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(54) Method and device for generating a person's portrait and communications terminal using 
the same 



(57) .The person's portrait generation device com- 
prises an im&ge input section (1) which picks up a two- 
dimensional image containing a person's face by using 
an image sensor. A head area extracting section (2) ex- 
tracts the head of the person from the image. A feature 
detection section (3) detects position of characteristic 
feature(s) of the face of that person. A face outline de- 
termining section. (4) determines a border between a 
face outline and a background. An image processing 
section *(5) generates a person's portrait in which the 
characteristic feature(s) of the face is emphasized. 
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Description 

FIELD OF THE INVENTION 

[0001] The present invention relates to a technology 
for generating an image of a person's portrait in which, 
from a two-dimensional image containing a person's 
face picked up by an image sensor, the feature of the 
subject person is emphasized to generate the person' s 
portrait or generate a deformed image of the person's 
portrait. This invention also relates to method and de- 
vice for communication, and a computer readable re- 
cording medium which stores a computer program 
which when executed on a computer realizes the meth- 
ods according to the present invention. 

BACKGROUND OF THE INVENTION 

[0002] In recent years, it has become possible to 
send/receive electronic mails not only on a personal 
computer but also on a portable terminal. Furthermore, 
not only documents, but also sender's portrait have 
been widely attached to the electronic mails. When cre- 
ating such a portrait by using a conventional method, it 
is necessary for the user to manually make selections 
from prepa r eri parts of several tens or more kinds, such 
as face outline pans, face pans and hairstyles, and com- 
bine them a!! together. In this case, a large number of 
combinations are available, and as a result, a great 
amount of workload is imposed on the user in an attempt 
to create a desired person's portrait. For this reason, 
there has been a strong demand for a device which can 
easily and automatically generate a person* s portrait 
using an image of that person acquired by a camera. In 
order tc creates person's portrait, such a device needs 
to automatically carry out a sequence of processes of 
extracting the face portion from the image, detecting 
eyes, mouth, etc., and replacing the desired parts with 
already-prepared desired templates. 
[0003] As one example of the related technique, Jap- 
anese Laid-Open Patent Application No. 255017/1998 
(Tokukaihei 1 0-25501 7) discloses a method and device 
for generating a person' s portrait. In this method, with 
respect to the face area detection, (R, G, B) values of 
skin color are preliminarily set to a specified range, and 
pixels having the corresponding color are marked, and 
clustered so as to carry out the detection, while the face 
parts detection is earned out by template matching proc- 
esses. However, with respect to the face area detection, 

(1) : the face color is influenced by illumination con- 
ditions, resulting In a difficulty in detecting the face 
area; in some cases; 

(2) the face color is not located within the above- 
mentioned range of the (R, G, B) values due to dif- 
ferences in human races, etc.; and 

(3) when -pixels 'having a color similar to the face 
color exists on the background, it is sometimes dif- 
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flcult to detect the face area. 

Moreover, with respect to the face parts posi- 
tion detection, conventional problems are that, 

(4) when the number of face-part templates is in- 
s creased so as to match various face parts contained 

in an image, the amount of calculations increases 
due to increased collation processes, resulting in 
degradation in the execution speed; and 

(5) a dark shadow tends to appear on the periphery 
of a face part in an image depending on illumination 
conditions, resulting in a difficulty in the template 
matching. 

SUMMARY OF THE INVENTION 

[0004] It Is an object of this invention to provide a per- 
son's portrait generation device which picks up a two- 
dimensional image that contains the person' s face by 
using an image sensor, analyzes characteristics fea- 
tures in the face from the picked image, and automati- 
cally generates an image in which characteristics fea- 
tures of the face are emphasized. It is another object of 
this invention to provide a person's portrait generation 
method for such a device. It is still another object of this 
invention to provide a communication terminal which in- 
corporates the person's portrait generation device ac- 
cording to this invention, and provide a communication 
method thereof. It is still another object of this invention 
a computer readable recording medium which stores a 
computer program which when executed on a computer 
realizes the methods according to the present invention. 
[0005] According to one aspect of this invention, two- 
dimensional images containing a person 1 s face are 
picked, area corresponding to head of the person is ex- 
tracted based on a differential image of the acquired 
two-dimensional images. After that, position of charac- 
teristic features of the face of that person are detected 
from the area corresponding to head of the person. 
Then, outline of the face is detected by detecting a bor- 
der between the face and the background of the image. 
Finally, the person's portrait In which the characteristics 
features of the person's face are emphasized is gener- 
ated based upon the acquired two-dimensional images 
and data corresponding to the area corresponding to 
head, position of characteristic features of the face, and 
the outline of the face. 

[0006] According to another aspect of this invention, 
a two-dimensional image containing a person's face is 
picked, area corresponding to head of the person is ex- 
tracted from the acquired two-dimensional image. After 
that, position of characteristic features of the face of that 
person are detected by dividing the head area thus ex- 
tracted into face part areas for respective face parts and 
finding projection data of binarized images for the re- 
spective face parts areas. Then, outline of the face is 
detected by detecting a border between the face and 
the background of the image. Finally, the person' s por- 
trait in which the characteristics features of the person' 
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Fig. 3 Is an explanatory drawing that shows a me- 
dian filter in accordance with the first embodiment 
of the present Invention. 

Fig. 4 is an explanatory drawing that shows a 
smoothing filter in accordance with the first embod- 
iment of the present invention. 
Fig. 5 is an explanatory drawing that shows the op- 
eration of a feature detection section in accordance 
with the first embodiment of the present invention. 
Fig. 6 is an explanatory drawing that shows the op- 
eration of a face outline determining section in ac- 
cordance with the firsl embodiment of Ihe present 
invention. 

Fig. 7 is a drawing that shows a specific example 
for a face portrait image that is formed based upon 
an input image by an image processing section in 
accordance with the first embodiment of the present 
invention. 

Fig. 8 is a flow chart indicating the steps in a per- 
son's portrait generation method in accordance with 
the first embodiment of the present invention. 
Fig. 9 is a flow chart indicating an image input step 
in accordance with the first embodiment of the 
present invention. 

Fig. 1 0 is a flow chart indicating a head area extract- 
ing step in accordance with the first embodiment or 
the present invention. 

Fig. 1 1 is a flow chart indicating a feature extracting 
step in accordance with the first embodiment of the 
present invention. 

Fig. 12 is a flow chart indicating a face outline de- 
termining step in accordance with the first embodi- 
ment of the present invention. 
Fig. 13 is a flow chart indicating an image process- 
ing step in accordance with the first embodiment of 
the present invention. 

Fig. 14 is an explanatory drawing that shows a spe- 
cific example for a face portrait image that is formed 
based upon an input image by an image processing 
section and a deforming method from the original 
image, in accordance with the second embodiment 
of the present invention. 

Fig. 15 is an explanatory drawing that shows a spe- 
cific example for a face portrait image that is formed 
based upon an input image by an image processing 
section and a deforming method from the original 
image, in accordance with the third embodiment of 
the present invention. 

Fig. 1 6 is an explanatory drawing that shows a com- 
munication terminal and a communication method 
in accordance with the fourth embodiment of the 
present invention. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 



s face are emphasized is generated based upon the ac- 
quired Two-dimensional image and data corresponding 
to the area corresponding to head, position of charac- 
teristic features of the face : and the outline of the face. 
[0007] According to still another aspect of this inven- 
tion, a two-dimensional image containing a person* s 
face is picked, and area corresponding to head of the 
person is extracted f rom the acquired two-dimensional 
image. After that, position of characteristic features of 
the face of that person are detected from the area cor- 
responding to head of the person. Then, outline of the 
face is detected by delecting a border between the face 
and the background of the Image. Finally, the person's 
portrait In which the characteristics features of the per- 
son's face are emphasized is generated based upon the 
acquired two-dimensional image and data correspond- 
ing to the area corresponding to head, position of char- 
acteristic features of the face, and the outline of the face. 
[0008] According to still another aspect of this inven- 
tion, a two-dimensional image containing a person's 
face is picked, and area corresponding to head of the 
person is extracted from the acquired two-dimensional 
image. After that, position of characteristic features of 
the f ac-3 for each of the face parts of that person are 
detected trom the area corresponding to head of the per- 
son. Then, outline of the face is detected by detecting a 
border between the lace and the background of the im- 
age. Finally : the person's portrait in which the charac- 
teristics features of the person's face are emphasized 
by changing the two dimensional image with respect to 
each o? the face parts. ■ 

[0009] Furthermore, a communication method and 
device according to still another aspect of this invention 
is capable of electronically transmitting or receiving the 
generated portrait using a communication unit. The 
communication device is, for, example, a cellular tele- 
phone or a PDA. . 

[0010] furthermore, a computer readable recording 
medium according to still another aspect of the present 
invention stores- a computer program which when exe- 
cuted realizes the methods according to the present in- 
vention. 

[001 1 ] Other objects and features of this invention will 
become apparent from the following description with ref- 
erence to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0012] • 

Fig 1 is a block diagram that shows an arrangement 
of a person's portrait generation device in accord- 
ance with the first embodiment of the present inven- 
tion. 

Fig; 2 isan explanatory drawing that shows the op- 
eration of a head area extracting section in accord- 
ance with the first embodiment of the present inven- 
tion. 



[0013] A first embodiment of the present invention will 
be explained first with reference to the accompanying 
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drawings. Fig. 1 is a block diagram that shows a con- 
struction of an image generation device in accordance 
with the first embodiment of the present invention. This 
device comprises five sections. That is, an image input 
section 1, a head area extracting section 2, a feature 
detection section 3 ; a lace outline determining section 
4 and an image processing section 5. 
[0014] For convenience of explanation, it will be as- 
sumed here that only one person is present in an image 
that has been picked up by an image sensor, and that 
although there is change in the position of the person 
Ihere is no movement in the background of the image. 
[0015] The image input section 1 including an image 
sensor picks up rwo-dimenslonal Images. The head ar- 
ea extracting section 2 analyzes a plurality of the two- 
dimensional Images picked up by the image input sec- 
tion 1 so that an area including the head of a person is 
extracts d. The feature detection section 3 analyzes the 
luminance values of the input images with respect to the 
area extracted by the head area extracting section 2 so 
that the positions of face parts such as the eyes and 
mouth are detected. Based upon the face parts posi- 
tions detected by the feature detection section 3, the 
face outline determining section 4 analyzes the color of 
a specific target area, such as the face color of the per- 
son, and based upon this analysis, determines the bor- 
der between the target area and the background. Based 
upon output information from the head area extracting 
section. 2 : the feature detection section 3 and the face 
outline determining ' section 4, the image processing 
section 5 creates an image in which the characteristic 
features of the target area is emphasized from the input 
images.'' 

[0016] Referring to Fig. 2A to Fig. 2K, a detailed ex- 
planation will be given of the operation of the head area 
extracting section 2 . In the first embodiment of the 
present invention, the head area extracting section 2 
calculales a differential image by using two images that 
have time-wise differences among the input images 
picked up by the image input section. In this case, in 
genera!;, it is preferable to use two images that virtually 
continue in terms of time. Moreover, in the differential 
image, pixel data are constituted by values obtained by 
binarizing the absolute value of a difference between 
corresponding pixei values. 

[0017] Fig. 2A to Fig. 2K are explanatory draws that 
show how the heacTarea of a person is detected by using 
the differential image calculated from the images picked 
up by the image input section 1 . Fig. 2A shows an image 
including the head of a person picked up by the image 
input section t. Fig. 2B is a differential image that has 
been binarized. In other words, a black pixel indicates 
a pixel in which there is any change in the pixel value, 
that is.ia pixel that has been moved. In this example, 
only one person moves; however, black pixels are 
sparsely mixed into the background due to noise con- 
tained in the image. Moreover, when any moving object 
is contained in the background, black pixels are also 
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mixed Into the background portion; however these 
black pixels are processed as noise when the main por- 
tion of the Image Is occupied by one person. 
[001 8] Next, from Fig. 2B, black pixels are retrieved 
s in the horizontal direction of the image. In other words, 
with respect to black pixels that are aligned along one 
horizontal line, the position of a black pixel is retrieved 
rightwards from the left end, or leftwards from the right 
end, and the position of the black pixel that has been 
10 first encountered is recorded. This process is repeated 
with respect to all the horizontal lines. Fig. 2C shows the 
results (hereinafter, referred lo as left side profile) of the 
retrieval rightwards from the left end in which the posi- 
tions of the black pixels that have been first encountered 
rs are connected from the upper end to the lower end. Fig. 
2D shows the results (hereinafter, referred to as right 
side profile) of the retrieval leftwards from the right end 
obtained in the same manner with respect to the posi- 
tions of the black pixels. 

[001 9] As can be seen in these cases, each of the two 
profiles has an irregular edge shape due to the black 
pixels that are noise components sparsely contained in 
the background. Therefore, prior to obtaining the outline 
of the head area, it is necessary to eliminate the irregular 
edge shape that forms noise components. 
[0020] In order to eliminate this irregular shape, a fil- 
tering process is preferably carried out. For example, a 
median filter in which the horizontal position in the right 
and left profile is represented as a length value from ihe 
end is applied so as to eliminate the noise components. 
[0021] Referring to Fig. 3. a brief explanation will be 
given of the median filter (center value filter). 
[0022] In the median filter, in general, n-number ot da- 
ta d[0], .... d[n - 1) are sorted in the order to sizes, and 
d[k) (0 ^ k ^ n - 1), which is located in the middle in the 
order to sizes, is selected as an output value. 
[0023] In Fig. 3, for example, with respect to re- 
number of column data d(1) Lo d(N). with respect lo ar- 
bitrary data d(k), a total of (2m + 1) number of data in- 
cluding m-number of proximity data before and after 
thereof are sorted in the order to sizes, and d(k) is re- 
placed by a value located in just the middle number (m 
+ 1) ; thus, the filtering process is realized. In this man- 
ner, the slope of data (contrast) is properly maintained. 
In other words, as shown in Fig. 3, when high values are 
continuously inputted as in the case of an (x) data group 
(from the fifth data to the tenth data from the top) before 
the application of the process, the contrast (gaps be- 
tween the values) is appropriately maintained as shown 
in the Figure indicating a state after the application of 
the process. 

[0024] In contrast, even when a high value is singly 
inputted as in the case of a (y) data group (from the fifth 
data from the bottom to the third data from the bottom) 
before the application of the process, this is eliminated 
by taking the center value of the proximity data as shown 
in the Figure indicating a state after the application of 
the process, thereby making it possible to eliminate the 
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noise. 

[0025] Fig. 2E and Fig. 2F respectively show the re- 
sults obtained by applying the median filter to the pro- 
files of Fig. 2C and Fig. 2D. However, as shown by the 
results of Fig. 2E and Fig. 2F, even the application of the 
median filter sometimes can not eliminate all the noise 
components depending on the degree of noise mixture. 
Therefore, the application of a smoothing filter to Fig. 
2E and Fig. 2F makes it possible to further eliminate the 
noise components. 

[0026] In the smoothing filter, in general, supposing 

lhal n-number of data d [0] d[n - 1 ] exist, for example, 

with respect to arbitrary profile data d[k], (d[k - 1] + d[k] 
+ d[k +■ i ])/3 is calculated, and the average value Is out- 
putted as d[k] 

[0027] Referring to Fig. 4, a brief explanation will be 
given of the smoothing filter. 

[0028] In Fig. 4, for example, in connection with re- 
number of column data d (1) to d (N), with respect to 
arbitrary data d(k), an average value of a total of (2m + 
1 ) number of data including m-number of proximity data 
before and after thereof are calculated to find an aver- 
age value, and d(k) is replaced by this value; thus, the 
filtering process is realized. In this manner, the data can 
be smoothed by using comparatively less amount of cal- 
culations. In otrferwords : as shown in Fig. 4, when high 
values pre continuously inputted as in the case of an (x) 
data group (Trom the fifth data to the tenth data from the 
top) before the application of the process, a smoothing 
process is available as a whole although the contrast 
becomes slightly dull as shown in the Figure indicating 
a state after the application of the process. 
[0029] Moreover, even when a high value is singly in- 
putted as in the case of a (y) data group (from the fifth 
data from the bottom to the third data from the bottom) 
before the application of the process, noise is clearly 
eliminated as shown in the Figure indicating a state after 
I he application of the process. 

[0030] Fig. 2G and Fig. 2H show the results of 
smoothing processes carried out on the right and left 
profiles of Fig 2E and Fig. 2F. These show that noise 
components are virtually eliminated so that the right end 
left outlines or the head are obtained. 
[0031] Next, an explanation will be given of a method 
in which a rectangular area enclosing the head area 
(hereinafter, referred to as a head rectangular shape) is 
determined by utilizing the right and left profiles shown 
in Fig. 2G and Rg. 2H. 

[0032] ■ Fi-g 2S shows a state in which left side profile 
6 of Fig: 2G and right side profile 7 of Fig. 2H are super- 
posed. Herej in Fig. 21. retrieval is carried out downward, 
starting fromt-he upperend of the horizontal line, with 
respect to the* right and left profiles so as to find the first 
crossing portion, and a horizontal line passing through 
the first crossing portion is defined as the upper side of 
the heed rectangular shape 8. Next, in order to deter- 
mine the left side' of the head rectangular shape 8, the 
retrieve:' is further continued downward with respect to 



the left side profile 6. A pixel position at which the dis- 
tance from the right end of the image first starts to de- 
crease is recorded, and a vertical line passing through 
this point is defined as the left side of the head rectan- 

5 gular shape 8, and recorded. With respect to the right 
side profile 7, the retrieval is made in the same manner 
so that a vertical line passing through the resulting pixel 
position is defined as the right side of the head rectan- 
gular shape 8, and recorded. 

io [0033] With respect to the remaining lower side of the 
head rectangular shape 8, a candidate position is deter- 
mined by utilizing the fact that the ratio of the width and 
the height of the head rectangular shape 8 of a general 
person Is virtually constant. More specifically, supposing 

is that the width is W, the height is H and the constant rep- 
resenting the average ratio is k, H/W ^ k is obtained. 
Here : the value of W is given by the distances of the 
right side and left side of the head rectangular shape 8 
that have already been determined, and the position 

20 (the distance from the upper end of the image) p of the 
upper side has already been determined: therelore, 
supposing that the candidate position of the lower side 
to be found is p* , p* (the distance from the upper end of 
the image) is calculated by the following expression: 

25 

p' = k x W + p. 

[0034] Fig. 2J shows a state in which the head rec- 
30 tangular shape 8 thus determined from the right and left 
profiles is displayed in a superposed manner. Finally, 
the head rectangular shape 8 is determined as a silhou- 
ette of a person to which movements of the person are 
reflected as shown in Fig. 2K that is obtained in a man- 
as ner so as to enclose the head. 

[0035] Next, an explanation will be given of the oper- 
ation of the feature detection section 3. Fig. 5A to Fig. 
5E is a drawing that explains an example in which lumi- 
nance values of an input image within the head rectan- 
40 gular shape 8 are analyzed so that the posilions of char- 
acteristic features of the eyes, nose, mouth, etc. are 
specified. 

[0036] Fig. 5A shows an example of an input image. 
Fig. SB shows a state in which the head rectangular 

^5 shape 8 extracted by the head area extracting section 
2 is displayed in a superposed manner. Fig. 5C shows 
areas 9 to 1 7 used for specifying rectangular areas that 
are analyzing ranges for identifying the positions o1 re- 
spective face parts that form characteristic features, and 

50 these areas are obtained by preliminarily finding relative 
positions of the respective face parts with respect to the 
head shape of a genera! person. 

[0037] In other words, with respect to the upper, lower, 
right and left sides of each of the rectangular shape ar- 
55 eas, the right eye is specified by points 14, 15, 10 and 
9, the right eye Is specified by points 14, 15. 9 and 13, 
the nose Is specified by 1 5, 1 6, 1 1 and 12, and the mouth 
is specified by 16, 17, 11 and 12. 
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[0038] Next, among the tace parts detections by the 
use of F'ig. 5D, for example, an explanation will be given 
of a method by which the position of the right eye Is de- 
tected. The detection area surrounded by 14, 15, 9 and 
1 0 is obtained by fcinarizing pixel values of the input im- 
age, and with respect to the pixel values, values greater 
than a threshold value are represented by 0 (while pix- 
els) and values other than this case are represented by 
1 (black pixels). In order to calculate the detection posi- 
tion (the center of the pupil) of the right eye by using the 
center-of-gravity position, projection data are obtained 
by adding values of respective pixels in the binarized 
image within the detection area for each row and each 
column, and the center-of-gravity position of each of 
these is calculated so that the horizontal position and 
vertical position of the right eye detection position are 
obtained. In these cases, 25 and 26 respectively repre- 
sent projection data in the vertical direction (for each col- 
umn) and projection data in the horizontal direction (for 
each row), the center-of-gravity positions are calculat- 
ed from these projection data. In the horizontal direction, 
27 is obtained as the center-of-gravity position, and in 
the vertical direction, 28 is obtained as such, with the 
result that the rrgr.t-eye detection position 19 shown in 
Fig. 5E »s obtained. 

[0039] Here, with respect to the horizontal and vertical 
positions, the positions, each providing the maximum 
value of the projection data, may be used instead of the 
center-of-gravity of the projection data so as to calculate 
these positions. 

[0040] In the same manner, the detection position 1 8, 
20 or 21 shown in Fig 5E for each of the left eye, nose 
and mouth is found by calculating the center of gravity 
from the binarized image within each of the rectangular 
shape ;*reas. 

[0041] The above explanation has been given of the 
eyes, nt>se and mouth serving as the characteristic fea- 
tures; however, in an attempt to improve the precision 
as a pe.son' s portrait, the detecting process may be 
carried out r>n olher parts elements, such as the eye- 
brows, Tront hairstyle, cheeks and ears. 
^[0042] Next, referring to Fig. 6 A to Fig. 6G, an expla- 
nation will be given of the operation of the face outline 
determfning section 4. In the present embodiment, the 
face nuvline determining section 4 calculates the RGB 
value of the average color of pixels in the proximity of 
each fa^e parts position by using information of the face 
parts position, detected by the head feature detection 
section^ based upon the input image of the image input 
section- 1 . and converts this value to a value on the HSV 
color system; thus, the RGB values of all the pixels of 
the inpi;t image are converted to the HSV values, and 
these values are plotted on the HSV color system. At 
this timg, an area centered on the HSV value of the av- 
erage color Is set on the HSV color system map, and 
the pixel data located within the area is again returned 
to the RGB color system, and plotted on the original in- 
put image. Based upon the results of these processes, 
;.. ■ ■ • v i-. •■ • . ' ' 



the face area is estimated. 

[0043] In this case, with respect to the conversion of 
the color system from the RGB values to the HSV val- 
ues, for example, a method disclosed in US. Paten! No. 
5 5586197 is used so as to realize the conversion. Here, 
the RGB color system uses a cubic shape having R : G, 
Bas its axes, whilethe HSV color system uses a cylinder 
shape as shown in Fig. 6A. With respect to the HSV val- 
ues, the value V (luminance) is represented by the 
io height from the bottom of the cylinder serving as the ref- 
erence as shown in Fig. 1A, the value S (the degree of 
saturation, the density of color) is represenled by the 
distance from the center axis C of the cylinder as shown 
In Fig. 6B, and H (the degree of chrominance, the kind 
15 of color) is represented by an angle from a line CR of a 
mapped red color with the values of G and B of the RGB 
values being set to 0 on a disc that is a cross-section of 
the cylinder. 

[0044] Fig. 6C shows an example in which upon con- 
version of the RGB values of an input image into HSV 
values, these values are plotted on the disc shown in 
Fig. 6B with differences in the value of V being ignored. 
[0045] Next, an explanation will be given of a system 
in which an area is provided on the HSV color system 
based upon the average color in the vicinity of the face 
parts position and the face area is specified based upon 
the color within the area. Fig. 6D and Fig. 6E show color 
judgment areas for extracting the face area, and 22 rep- 
resents a position obtained when the average color 
(RGB value) in the vicinity of the face parts position is 
converted into an HSV value. As shown in Fig. 6D, the 
shape of the color judgment area may be set as an el- 
liptical shape area 23 with a line CX serving as a major 
axis, when a crossing point of the circumference and the 
extended line of the line connecting the average color 
position 22 and the center C is set as X, or as shown in 
Fig. 6E, a delta shape area 24 may be set. In either of 
the cases, pixels within the area including Ihe average 
color position 22 and having colors similar to this can be 
specified as face area candidates. 
[0046] An explanation will be given of a system in 
which, by analyzing the positions of the pixels serving 
as face area candidates that have been determined as 
described above, the face outline, especially, the border 
line in the vicinity of the jaw, is estimated. The determin- 
ing process of the face outline in the vicinity of the jaw 
is important in creating a person's portrait. In other 
words, the case in which the right and left smoothed pro- 
files 6 and 7 are simply used as the face outline in the 
vicinity of the jaw fails to provide a person's portrait suf- 
ficiently representing the characteristic features ol the 
face, and in order to obtain a superior person's portrait, 
it is necessary to individually determine the face outlines 
in the vicinity of the jaw. 

[0047] First, with respect to a head outline 29 shown 
in Fig. 6F, the Inside of the head rectangular shape 8 is 
analyzed by the above-mentioned specifying method of 
the face area by using the differential image used in the 
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area extracting section 2, partial elliptical shapes of four 
corners are calculated and set in amannerso as to sur- 
round this specified face portion : and one portion of 
each ol the right and left sides of the head rectangular 
shape 6 is further added to the partial elliptical shape so 
that the outline of the face including a jaw border line 30 
is formed. 

[0048] More specifically, on the lower side surrounded 
by the tight and left side of the head rectangular shape 
8. that is, within the semi-elliptical area of the head out- 
line 29 , the rate of the face area candidate pixels, ex- 
plained before, is calculated. In this case, the semi-el- 
liptical shapes of the head outline 29 are contracted to- 
ward the respective centers at the same curvature until 
the calculated rate has exceeded a predetermined 
threshold value, thereby determining the Jaw border line 
30. Fig. 6G shows an example of the estimated results 
of the jaw border line 30 obtained by applying the above- 
mentioned technique to an actual image. 
[0049] Lastly, an explanation will be given of the op- 
eration ol the image processing section 5. In the first 
embodiment, the Image processing section 5 generates 
a person* s portrait of a face image contained in an orig- 
inal im^ge by using the head rectangular shape 8 de- 
tected by the head area extracting section 2, the posi- 
tions of the eyes 18, 19 and the mouth position 21 de- 
tected by the feature detection section 3, the head out- 
line 29 .tetected by the face outline determining section 

4 and the estimated jaw border line 30. Fig. 7 shows an 
example of a person's portrait image generated by the 
image processing section 5. In this image, the upper half 
of the h?jad outline 29 and the portion surrounded by the 
jaw border line 30 thereof are extracted from the input 
image to forma face image, and after having been sub- 
jected to an ericje-emphasizing process, this is binarized 
to form «% line imnge ; At this time, the inside of the face 
area may be painted using the average color to which 
the face-; color calculated by the face outline determining 
section 4 is reflected; thus, it is possible to obtain the 
person's porirai! image as a color image. 

[0050] = The above-mentioned description has dis- 
cussed the construction of the person's portrait gener- 
ation device, and the following description will briefly dis- 
cuss the person's portrait generation method by refer- 
ence to a flow chart that summarizes its sequences of 
processes. • 

[0051] Fig/8 is a f low chart that shows the person's 
portrait generation method of the present invention cor- 
responding to Fig. T. Steps 101 to 105 shown in this di- 
agram £re peitormed respectively by the sections 1 to 

5 shown in Fig. 1; It is to be noted that the sequence of 
steps 1T.3 and 104 may be reversed. 

[0052] Figiy&to Fig. 1 2 are flow charts showing detail 
processing inthe steps 101 to 104 shown in Fig. 8. 
[0053] : m the image input step 101 shown in Fig. 9, 
first, two image buffers in which input images are stored 
are initializer*.: Next, image input is started. Then, image 
data ar*» respectively stored in the two image buffers, 



and If no other pickup image exists, the image pickup 
process is completed, and the image data are outputted. 
[0054] In a head area extracting step 102 shown in 
Fig. 1 0, first, based upon the image data outputted from 

s the two image buffers, calculations are made to gener- 
ate a differential image between binarized frames. Next, 
the differential image between the binarized frames is 
analyzed to extract a head rectangular shape. 
[0055] In a feature detection step 103 shown in Fig. 

10 11 1 first, based upon the rectangular area extracted by 
the head area extracting step 1 02 and relative positions 
of the respective face parts wilh respect lo Ihe head 
shape that have been preliminarily provided : an analy7- 
Ing range used for detecting a position of each of the 

15 face parts is determined. Next, the image within the an- 
alyzing range is analyzed for each of the face parts so 
that the center position of each of the face parts is de- 
tected. 

[0056] In the face outline determining step 1 04 shown 
20 jn Fig. 12, first, based upon the image inputted from the 
image input step 1 01 , the average color of the face is 
calculated by using RGB pixel data in the vicinity of each 
of the face part positions detected by the feature detec- 
tion step 1 03. Then, within the rectangular area extract- 
's ed in the head area extracting step 102, pixels having 
RGB values close to the average color of the face cal- 
culated in the first step in a specified range are recorded 
as skin-color pixels. Next, the skin-color pixels thus re- 
corded are analyzed to determine the face outline. 
30 [0057] The above-mentioned image generation cal- 
culating method from the picked-up image is realized by 
an image generation program, and the program is re- 
corded in a recording medium that can be read by a 
computer, etc., and supplied. 
35 [0058] The first embodiment has exemplified the case 
in which, in the head area extracting section 2, in order 
to obtain a differential image by using two time-wise con- 
secutive images among input images picked up Ihe im- 
age input section 1 , the absolute value of the difference 
40 of the corresponding pixels is binarized. However, a 
method other than this, for example, a method in which 
an image is subjected to a process for extracting the out- 
line of a person, such as an edge analysis, so as to ex- 
tract the outline including the head and shoulder as de- 
45 scribed in the first embodiment, may be used , and there- 
after, in the same manner as described above, a per- 
son's portrait may be generated. 

[0059] Moreover, the first embodiment has exempli- 
fied the case in which, in the head area extracting sec- 

so tion 2, a differential image is calculated by using two 
time-wise consecutive images among input images 
picked up the image input section 1. However, for ex- 
ample, an image corresponding to one frame picked up 
by the image input section! is used, and supposing that 

55 only one person Is Included In the image and that the 
background has a uniform color, a new image is formed 
by shifting this image up and down or right and left by 
several pixels, and to these two images, the same proc- 
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ess as the differential image process of the head area 
extracting section 2 described in the first embodiment 
may be applied: thus, it is possible to obtain the same 
results. 

[0060] As described above, in the first embodiment, 
based upon two- dimensional images picked up by the 
image input section 1 , a characteristic teat u re from the 
person's face is detected by analyzing the luminance of 
pixel values, and the person's portrait, etc., that is color- 
ed based upon the calculated average color in the vicin- 
ity of the characteristic feature is formed, that is, an im- 
age emphasizing the c haracteristic feature of the pickup 
subject is created. Therefore, without the need of tem- 
plate matching, the characteristic feature of the face 
parts, etc. are automatically detected, and the skin color, 
etc. of the face Is caiculated from the plcked-up image. 
Consequently, it Is not necessary to specify the color be- 
forehand, and with respect to the background, in the 
case o? a virtually stationary background, the pickup 
subject, such as a person, can be detected stably inde- 
pender ' of the kinds of the background, with the result 
that an Image in which the characteristic feature of the 
pickup subject are emphasized, such as a person's por- 
trait formed by the resuits of picked-up images, can be 
generated in a stable manner. 

[0061] A second embodiment of the present invention 
will be explained next. The arrangement of a person's 
portrait genemtinn device in accordance with the sec- 
ond embodimR,':! )s the same as that of the first embod- 
iment. !n the second embodiment, in addition to the 
functions explained in the first embodiment, in the image 
processing section 5 ; the results of the detection of each 
of the face partspositions, such as the eyes, the mouth, 
etc., made by the- feature detection section 3 are utilized 
so as to deform tne person's portrait derived from a face 
image, thereby forming a deformed face image repre- 
senting emotional changes, such as a smiling face and 
a cryind face. - 

[0062] Fig. 14A to Fig. 14F are examples that show 
such deformations of a person's portrait. By utilizing a 
portrait shown in Fig; 1 4A (which is the same as the por- 
trait shown rn fig: 7), the person's portrait is deformed 
based i;p0n the positions of the eyes and mouth, there- 
by obtaining the results shown in Fig. 14B to Fig. 14F. 
These aire examples of the person's portrait deforma- 
tions. Fig. 14B shows a'smiling face, Fig. 14C shows a 
crying face, Pig.. 14D shows an angry face, Fig. 14E 
shows a fat face, and Fig. 14F thin face. 
[0063] Next, an explanation will be given of specific 
sequences of the face deformations. First, the face de- 
formations shown in* Fig. 14B to Fig. 14D are obtained 
by deforming a person's portrait image in accordance 
with directions of arrows determined based upon the po- 
sitions af the eyes and the position of the mouth shown 
in Fig. 14G to Fig. 141 respectively. Moreover, the face 
deformations shewn in Fig. 14E and Fig. 14F are ob- 
tained *?y defomiing the person's portrait Image so as to 
expand and shrink in the directions of arrows deter- 



mined based upon the jaw line, as shown in Fig. 14J to 
Fig. 14K. 

[0064] As described above, a person's portrait is gen- 
erated based upon an input image, and it is deformed 

5 based upon the detection positions of the face parts, 
thereby forming the person's portraits representing 
emotional changes such as a crying face and a smiling 
face; thus, it becomes possible to automatically gener- 
ate such images without the necessity of template se- 

10 lection by the user. 

[0065] Next, Fig. 13 is a flow chart showing steps in 
the image processing step 105. 

[0066] First, based upon an input image within the 
face outline calculated by the face outline determining 

'5 step 104, a person's portrait corresponding to Fig. 7 is 
generated. Next, after discriminations have been made 
as to whether or not an insertion of any of the face parts 
is required or whether or not a deformation of any of the 
face parts is required, a desired person* s portrait image 

20 is obtained. 

[0067] The above-mentioned image generation cal- 
culating method from the picked-up image is realized by 
an image generation program, and the program is re- 
corded in a recording medium that can be read by a 

25 computer, etc., and supplied. 

[0068] In the second embodiment, the explanation 
has been given of the deformations of a person's portrait 
image by the use of deformations of the face parts. How- 
ever, for example, the face parts from an original image 

30 may be used, and the original image may be partially 
deformed based upon the detection positions of the face 
parts so as to provide face deformations representing 
various emotions. Moreover, in the second embodi- 
ment, the explanation has been given of the case in 

35 which the deformations are made based upon the posi- 
tions of the eyes and the position of the mouth. However, 
characteristic features other than these, for example, 
the positions of the nose and the ears, may be used as 
the references on which the deformations are made. 

^o Furthermore, in the second embodiment, wiLh respect 
to the deforming method for emotional face images, the 
explanation has been given by reference to Fig. 14G to 
Fig. 1 4K. However, the present invention is not intended 
to be limited by this method, and any known method may 

*s of course be used to provide the deformations. 

[0069] A third embodiment of the present invention 
will be explained next. The arrangement of a person's 
portrait generation device in accordance with the third 
embodiment is the same as that of the first embodiment. 

so in the third embodiment, in addition to the functions ex- 
plained in the first embodiment, in the image processing 
section 5, other face parts are put at the face parts po- 
sitions, such as the eyes, the mouth, etc., detected by 
the feature detection section 3, in a replacing manner, 

55 so as to form another face image different from the per- 
son's portrait image shown in the first embodiment. 
[0070] Fig. 15A to Fig. 15E are drawings that explain 
how already prepared desired face parts are inserted to 
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the positions of the eyes and mouth of the person's por- 
trait image shown in Fig. 7 to form another face image. 
Fig. 15A shows a case in which asterisk marks are in- 
serted at the positions of the eyes and mouth. Further, 
when the asterisk mark corresponding to the right eye 
is replaced with an image of a right eye shown in Fig. 
15B,the asterisk mark corresponding to the left eye is 
replaced with an image of a left eye shown in Fig. 15C, 
and the asterisk mark corresponding to the mouth is re- 
placed with en image of a mouth shown in Fig. 15D, then 
the portrait shown in Fig. 15E is obtained. 
[0071] As described above, in a person's portrait gen- 
erated from an input image, other face parts images are 
inserted to the face parts positions automatically detect- 
ed, thereby forming another face image. Thus, it is pos- 
sible to omit the user's manually specifying operations 
related to parts positions, and also to automatically gen- 
erate various face images by preparing a number of 
such face parts. 

[0072] A fourth embodiment of the present invention 
will be explained next. The fourth embodiment exempli- 
fies a case in which the aforementioned person's portrait 
generation device or person's portrait generation meth- 
od is applied to a communication terminal. 
[0073] Fig. 16 is a drawing that exemplifies a case in 
which tne present invention is applied to a mobile tele- 
phone 31 serving fts a communication terminal. As 
shown ; n Fig. 1 6A, a face image is picked up by a small- 
size camera 32 attached to the mobile telephone 31 . 
Then the impge is subjected to a face parts position de- 
tection emd a person's portrait generating process to ob- 
tain theoersnn's portrait shown in Fig. 1 6B. This portrait 
(data ot the portrait) may be then transmitted to a de- 
sired destination: On the other hand, the portrait shown 
in Fig. 1SB may be deformed as shown in Fig. 16C. The 
portrait may be deformed to express person's emotions 
such as cry, smile, anger and joy, or expressing a fat 
face or X tnin race, thereby making il possible to transmit 
the resulting person's portrait data. Moreover, face parts 
position- information may be detected from the portrait 
shown in Fig. 16B, and the face parts may be exchanged 
with other parts or symbols as shown in Fig. 16D. The 
portraits shown in Fig. 16C or Fig. 16D may be trans- 
mitted to desired destination. 

[0074]- The "following description will propose exam- 
ples of'methods for actually using these person's por- 
traits that are effectively used at the time of communi- 
cation, such as mail exchange. For example, a setting 
may be made ir» : Which, on a waiting screen at the time 
of mai I deceiving: the user* s portrait is displayed with 
changes in erriolions, such as crying or smiling, depend- 
ing on the senders; thus, it is possible to immediately 
judge who sent a mail or what kind of person sent a mail. 
[0075] 'i- in another example of the application, mail 
senders' portraits are preliminarily registered, and a set- 
ting is made so -that, upon receipt of a mail, the corre- 
sponding sender's portrait is displayed; thus, it is possi- 
ble to immediately judge who sent the mail. Moreover, 



even when the receiver Is not an acquaintance, sending 
the sender's portrait may make it possible to provide a 
friendly mood. 

[0076] Furthermore, mail subjects may include items 

5 such as "sad information" and "good information", and 
a deformed person's portrait expressing a crying slate, 
smiling state, etc. is provided on the receivers side; 
thus, it becomes possible to immediately judge the con- 
tents of the mail by the expression of the portrait. 

io [0077] Here, the person's portrait formed by the 
present invention may be utilized as various indicators 
for mobile communication apparatuses, elc. For exam- 
ple, a setting may be made so that when the battery is 
running out : the user's thin portrait is displayed: thus, it 

15 is possible to easily know the exchanging time or charg- 
ing time for the battery. Moreover, when unread received 
mails have been accumulated, the receiver's fat (ace 
may be displayed so that it is possible to easily know 
the degree of accumulated mails. Furthermore, when di- 

so rect sun light enters the screen, a dazzled face may be 
displayed thereon so thatthe corresponding informalion 
is easily given. 

[0076] With respect to other different applications, 
when a game, etc., is played, upon achievement of a 

25 high mark, a smiling face may be displayed while upon 
receipt of a low mark, a crying face may be displayed; 
thus, the game becomes more enjoyable. Moreover the 
receiver's portrait may be combined with a transmitted 
person's portrait to have fun. Furthermore, parts such 

30 as beard, wigs, etc. may be preliminarily prepared., and 
these may be combined with a person's portrait to have 
fun. Furthermore, an angry face may be colored in red, 
or tear parts, preliminarily prepared separately may be 
added to a crying face to have fun. 

35 [0079] The above-mentioned communication termi- 
nal is not intended to be limited to the mobile telephone, 
and any communication terminal may be used as long 
as il can transmit and receive person's portrait images. 
For example, it can be applied to PDAs (Personal Digital 

40 Assistant), notebook PCs, television telephones. 

[0080] Moreover, in accordance with the person's por- 
trait generation device of the present invention, person's 
portraits may be attached to electric mails, etc. used in 
mobile telephones and personal computers. Alterna- 

45 tiveiy in a video conference, etc., using cameras be- 
tween remote places, instead of showing real images of 
the participants' faces, the participants* portrait images 
automatically generated may be given on the screen, 
and in response to audio data in the discussion, the 

so mouth portions of the portraits may be moved; thus, the 
participants are allowed to have realistic feelings, while 
avoiding a feeling of oppression the users might have 
due to the appearance of their actual face on the screen. 
[0081] Since the present invention is arranged and 

55 has sequences as described above, it provides the fol- 
lowing effects: 

[0082] The person' s portrait generation device in ac- 
cordance with the present invention is provided with the 
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image Input section, the head area extracting section, 
the feature detection section, the face outline determin- 
ing section and the image processing section; therefore, 
without the need of template matching, the characteris- 
tic features ot the face parts, etc. are automatically de- 5 
tected end the skin color, etc. of the face is calculated 
from the picked-up image so that it is not necessary to 
specify the color beforehand. Moreover, with respect to 
the background, in the case of a virtually stationary 
background, the pickup subject, such as a person, can io 
be detected stably independent of the kinds of the back- 
ground with the result that an image in which the char- 
acteristic features of the pickup subject are emphasized, 
such as a person's portrait formed by the results of 
picked-'jp images, can be generated in a stable manner, is 
[0083] Moreover the communication terminal of the 
present invention is provided with the image input sec- 
tion, the head area extracting section, the feature detec- 
tion sec: ion. the face outline determining section and the 
image processing section, and the person' s portrait 20 
thus formed is transmitted and received through the 
communication unit; therefore, without the need of tem- 
plate rr..itchir»$. *he characteristic features of the face 
parts, gtc. are automatically detected and the skin color, 
etc. of the face is calculated from the picked- up image & 
so that il is net necessary to specify the color before- 
hand. Moreover with respect to the background, in the 
case ov a virtuaily stationary background, the pickup 
subject, such as a person, can be detected stably inde- 
pendent of the kinds of the background, with the result 3Q 
that an Image in which the characteristic features of the 
pickup subject are emohasized. such as a person's por- 
trait fonned by the results of picked-up images, can be 
generated in a stable manner, and the person's portrait 
thus formed is utilized so as to inform the user of the zs 
transmission and receipt and is also utilized as an indi- 
cator fcr the communication terminal so that it becomes 
possible lo carry out pleasant communications while 
having fun. 

[0084] Th ^ person' s portrait generation method of the *0 
present-invention is provided with the image Input step, 
the he&d arc^a extracting step, the feature detection se- 
quence, the fece outline determining sequence and the 
image processing sequence; therefore, without the 
need ovtemp'eta matching, the characteristic features 45 
of the face parts; etc. are automatically detected and the 
skin co'or. etc. -of ;he face is calculated from the picked- 
up imaoe so that \X is not necessary to specify the color 
beforehand. Moreover, with respect to the background, 
in the case of a virtually stationary background, the pick- so 
up subject, such as a person, can be detected stably 
independent of the kinds of the background, with the re- 
sult that an image in which the characteristic features of 
the pickup subject errc emphasized, such as a person* 
s portrait formed by the results of picked-up images, can ss 
be generated in a stable manner. 

[0085] Moreover/ the communication method of the 
present-invention is provided with the image input step, 



the head area extracting step, the feature detection 
step, the face outline determining step : the image 
processing step and the step which, upon receipt of in- 
formation, displays the person's portrait thus formed so 
as to inform the user of the receipt of the information; 
therefore, without the need of template matching, the 
characteristic features of the face parts, etc. are auto- 
matically detected and the skin color, etc. of the face is 
calculated from the picked-up image so that it is not nec- 
essary to specify the color beforehand. Moreover, with 
respect to the background, in the case of a virtually sta- 
tionary background, the pickup subject, such as a per- 
son, can be detected stably independent of the kinds of 
the background, with the result that an image in which 
the characteristic features of the pickup subject are em- 
phasized, such as a person's portrait formed by the re- 
sults of picked-up images, can be generated in a stable 
manner, and the person's portrait thus formed is utilized 
so as to inform the user of the transmission and receipt 
so that it becomes possible to carry out pleasant com- 
munications while having fun. 

[0086] Furthermore, the recording medium of the 
present invention in which the person's portrait genera- 
tion program is recorded is provided with: the head area 
extracting program for extracting a head area of a per- 
son from a differential image of a plurality of two-dimen- 
sional images picked up by an image input section, the 
feature detection program for detecting positions of 
characteristic features of the person* s portrait by divid- 
ing the head area into face part areas for respective face 
parts, finding projection data of binarized images for the 
respective face parts areas and detecting the position 
of each of the face parts based upon a center-of-gravity 
position of the projection data, the face outline determin- 
ing program for determining a border between a face 
outline and a background within the head area by spec- 
ifying a skin color area of the two-dimensional image, 
and the image processing program for creating a per- 
son's portrait in which the characteristics in the person's 
face is emphasized by changing Ihe two-dimensional 
image with respect to each of the face parts; therefore, 
the person's portrait thus generated is read by a com- 
munication terminal, etc. capable of transmitting and re- 
ceiving images so that it is utilized so as to inform the 
user of the transmission and receipt, and is also utilized 
as an indicator for the communication terminal. 
[0087] Although the invention has been described 
with respect to a specific embodiment for a complete 
and clear disclosure, the appended claims are not to be 
thus limited but are to be construed as embodying all 
modifications and alternative constructions that may oc- 
cur to one skilled in the art which fairly fall within the 
basic teaching herein set forth. 



Claims 

1 . A person's portrait generation device comprising: 
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an image input section (1 ) which picks up a two- 
dimensional image containing a person's face 
by using an image sensor; 
a head area extracting section (2) which ex- 
tracts a head area from a differential image of 5 
a plurality of the two-dimensional images 
picked up by the image input section (1) ; 
a feature detection section (3) which detects 
position of characteristic features of the face 
within the extracted head area; 10 
a face outline determining section (4) which de- 
termines a border between a face outline and 
a background within the head area; 
an image processing section (5) which gener- 
ates a person's portrait in which the character- 1 $ 
istics in the person* s face Is emphasized based 
upon the two-dimensional image by using data 
acquired in the head area extracting section (2), 
the feature detection section (3) and the face 
outline determining section (4). so 



2. The-; person's portrait generation device according 
to ciaim 1 , 

wherein in the head area extracting section 
(2). the outline of the head area is obtained by com- 25 
binmg a right-side profile and a lift-side profile of the 
diffarential image. 

3. The person's portrait generation device according 

to claim 2. 30 

wherein i.he outline of the head area is ob- 
tained by eliminating noise components of the right- 
side profile and left-side profile through filtering. 

4. The person's portrait generation device according 35 
to claim 2, . 

wherein the head area extracting section (2) 
provides the head area obtained by a head rectan- 
gular shape in which a left side, a right side and an 
upper cidof/B determined by the outline of the head 40 
area and a iowerside is determined by a shape ratio 
constant of a general person's head. 

5. A person's portrait generation device comprising: 

45 

an image input section (1) which picks up a two- 
dimensional image containing a person's face 
by using an image sensor; 

■ a head area extracting section (2) which ex- 
tracts a riead area from the image picked up by so 

*"■ the imago input section (1); 
a feature detection section (3) which detects 
position of characteristic features of the face by 

•dividing the head area thus extracted into face 

• part areas *or respective face parts and finding ss 
projection data of binarized images for the re- 
speCtlve face parts areas; 
a face outline determining section (4) which de- 



termines a border between a face outline and 
a background within the head area: and 
an Image processing section (5) which gener- 
ates a person's portrait in which the character- 
istics in the person* s face is emphasized based 
upon the two-dimensional image by using data 
acquired in the head area extracting section (2), 
the feature detection section (3) and the face 
outline determining section (4). 

6. The person's portrait generation device according 
to claim 5, 

wherein the face parts area is determined by 
relative positional information of the face parts with 
respect to the head area preliminarily found 

7. The person's portrait generation device according 
to claim 5, 

wherein the position of characteristic features 
of the face is detected based on a position of a cent- 
er-of -gravity of the projection data. 

8. The person's portrait generation device according 
to claim 5, 

wherein the position of characteristic features 
of the face is detected by based on a position of a 
maximum value of the projection data. 

9. A person's portrait generation device comprising: 

an image input section (1 ) which picks up a two- 
dimensional image containing a person's face 
by using an image sensor; 
a head area extracting section (2) which ex- 
tracts a head area from the image picked up by 
the image input section (1); 
a feature detection section (3) which detecls 
position of characteristic features within the ex- 
tracted head area; 

a face outline determining section (4) which de- 
termines a border between a face outline and 
a background within the head area by specify- 
ing a skin color area of the two-dimensional im- 
age; 

an image processing section (5) which gener- 
ates a person's portrait in which the character- 
istics in the person' s face is emphasized based 
upon the two-dimensional image by using data 
acquired in the head area extracting section (2). 
the feature detection section (3) and the face 
outline determining section (4). 

10. The person's portrait generation device according 
to claim 9, 

wherein the face outline determining section 
(4) determines the outline of the jaw. 

11. The person's portrait generation device according 
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to claim 9, 

wherein in the face outline determining sec- 
tion (4), the skin color area is determined by con- 
verting an inputted RGB value to an HSV value. 

12. The person's portrait generation device according 
tociaim9, 

wherein, with respect to an average color in 
the vicinity of the characteristic features of the face, 
areas having similar colors are used as face area 
candidates. 

13. A person's portrait generation device comprising: 

an image input section (1 ) which picks up a two- 
dimensional Image containing a person's face 
by using an image sensor; 
a head area extracting section (2) which ex- 
tracts a head area from the image picked up by 
the image input section (1); 
a feature detection section (3) which detects 
position of characteristic features of the face for 
each ol the face parts within the head area that 
has been extracted; 

& face outline determining section (4) which de- 
termines a border between a face outline and 
a background within the head area; and 
an image processing section (5) which gener- 
ates a person's portrait in which the character- 
istics in the person's face is emphasized by 
changing the two-dimensional image with re- 
speci to. each of the face parts. 

14. The person's portrait generation device according 
to cJaim 13, 

wherein the image processing section (5) ex- 
presses a person* s emotions by changing the 
shape of s partial image at each of an eye portion, 
a roisa pd-tion and a mouth portion that are the 
chiiracl'inslic features of the face. 

15. The person's portrait generation device according 
tociaim13, 

wherein the image processing section (5) re- 
places partial images at an eye portion, a noise por- 
tion and a-mouth portion that are characteristic fea- 
tures-of the face. 

16. A communication terminal comprising: 

an image input section (1 ) which picks up a two- 
dimensional image containing a person's face 
by using an. image sensor; 
a head area extracting section (2) which ex- 
tracts a head area from a differential image of 
: a plurality of the two-dimensional images 
^picked up by the image Input section (1); 
a feature detection section (3) which detects 



position of characteristic features within the ex- 
tracted head area; 

a face outline determining section (4) which de- 
termines a border between a face outline and 

5 a background within the head area; and 

an image processing section (5) which gener- 
ates a person's portrait in which the character- 
istics in the person's face is emphasized based 
upon the two-dimensional image by using data 

10 acquired in the head area extracting section (2), 

the feature detection section (3) and the face 
outline determining section (4), 

wherein the resulting person's portrait is 
*5 transmitted and received through a communication 

unit. 

17. The communication terminal according to claim 16, 

wherein in the head area extracting section 
2° (2), the outline of the head area is obtained by com- 

bining a right-side profile and a lift-side profile of the 
differential image. 

18. The communication terminal according to claim 1 7 : 
25 wherein the outline of the head area is ob- 
tained by eliminating noise components of the i ighj- 
side profile and left-side profile through filtering 

19. The communication terminal according to claim 1 7. 
30 wherein the head area extracting section (2) 

provides the head area obtained by a head rectan- 
gular shape in which a left side, a right side and an 
upper side are determined by the outline of the head 
area and a lower side is determined by a shape ratio 
3s constant of a general person's head. 

20. A communication terminal comprising: 

an image input section (1 ) which picks up a two- 

40 dimensional image containing a person's face 

by using an image sensor; 
a head area extracting section (2) which ex- 
tracts a head area from the image picked up by 
the image input section (1); 

45 a feature detection section (3) which detects 

position of characteristic features of the face by 
dividing the head area thus extracted into face 
part areas for respective face parts and finding 
projection data of binarized images for the re- 

so spective face parts areas; 

a face outline determining section (4) which de- 
termines a border between a face outline and 
a background within the head area; and 
an image processing section (5) which gener- 

55 ates a person's portrait in which the character- 

istics in the person's face is emphasized based 
upon the two-dimensional image by using data 
acquired in the head area extracting section (2), 
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the feature detection section (3) and the face 
outline determining section (4), 

wherein the resulting person's portrait is 
transmitted and received through a communication s 
unit. 



27. The communication terminal according to claim 24, 
wherein determining the skin color area in the 
face outline determining section (4), with respect to 
an average color in the vicinity of the characteristic 
features of the face, areas having similar colors are 
used as face area candidates. 



21 . The communication terminal according to claim 20, 

. wherein the face parts area is determined by 
relative positional information of the face parts with io 
respect to the head area preliminarily found. 

22. Th« communication terminal according to claim 20, 

wherein the position of characteristic features 
of the face is detected based on a position of a cent- is 
er-of-graviry of the projection data. 

23. The communication terminal according to claim 20, 

wherein the position of characteristic features 
of the face is detected by based on a position of a 20 
maximum value of the projection data. 

24. A communication terminal comprising: 

an image input section (1 ) which picks up a two- 25 
dimensional image containing a person's face 
by using an image sensor; 
a head area extracting section (2) which ex- 
tracts a need area from the image picked up by 
the image input section (1); 30 
a feature detection section (3) which detects 
position of characteristic features within the ex- 
tracted head area; 

. a tace outline determining section (4) which de- 
termines a border between a face outline and 35 
a background within the head area by specify- 
ing a skin color area of the two-dimensional im- 
age; and 

an image processing section (5) which gener- 
ales a person's portrait in which the character- *o 
istics in the person' s face is emphasized based 

• upon ihre two-dimensional image by using data 
acquired in the head area extracting section (2), 
the feature detection section (3) and the face 

? outline determining section (4), <s 

wherein the resulting person' s portrait is 
transmitted and received through a communication 
unit. ! : 

50 

25. The communication terminal according to claim 24, 

wherein the face outline determining section 
(4) determines the outline of the jaw. 

26. The* communication terminal according to claim 24, ss 

. - wherein in the face outline determining sec- 
tion (4). "the skin color area is determined by con- 
verting an inputted RGB value to an HSV value. 



28. A communication terminal comprising: 

an image input section (1 ) which picks up a two- 
dimensional image containing a person's face 
by using an image sensor; 
a head area extracting section (2) which ex- 
tracts a head area from the image picked up by 
the image input section (1); 
a feature detection section (3) which detects 
position of characteristic features of the face for 
each of the face parts within the head area that 
has been extracted; 

a face outline determining section (4) which de- 
termines a border between a face outline and 
a background within the head area; and 
an image processing section (5) which gener- 
ates a person's portrait in which the character- 
istics in the person's face is emphasized by 
changing the two-dimensional image wilh re- 
spect to each of the face parts, 

wherein the resulting person' s ponraii is 
transmitted and received through a communication 
unit. 

29. The communication terminal according to claim 28. 

wherein the image processing section (5) ex- 
presses a person's emotions by changing the shape 
of a partial image at each of an eye portion, a noise 
portion and a mouth portion that are the character- 
istic features of the face. 

30. The communication terminal according 10 claim 28, 

wherein the image processing section (5) re- 
places partial images at an eye portion, a noise por- 
tion and a mouth portion that are characteristic fea- 
tures of the face. 

31. A person's portrait generation method comprising: 

an image input step of picking up a two-dimen- 
sional image containing a person's face by us- 
ing an image sensor; 

a head area extracting step of extracting a head 
area from a differential image of a plurality of 
the two-dimensional images picked up in the 
image input step; 

a feature detection step of detecting position of 
characteristic features within the extracted 
head area; 

a face outline determining step of determining 
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a border between a face outline and a back- 
ground within the head area; and 
an Image processing step of creating a per- 
son's portrait in which the characteristics in the 
person's face are emphasized based upon the 
two-dimensional image by using output data 
obtained in the head area extracting step, the 
. feature detection step and the face outline de- 
termining step. 

32. The person' s portrait generation method according 
lo claim 31 , 

wherein in the head area extracting step, the 
outline of ;i*e head area Is obtained by combining a 
right-side profile and a lift-side profile of the differ- 
ential Image. 

33. The person' s portrait generation method according 
to claim 32, 

wherein the outline of the head area is ob- 
tained by eliminating noise components of the right- 
side profile and left-side profile through a filtering 
step. 

34. The person's portrait generation method according 
to ciaim 32, 

wherein the head area extracting step pro- 
vides the head area that is obtained by a head rec- 
tangular shape in which a left side, a right side and 
an upper .side are determined by the outline of the 
hefid area and a lower side is determined by a 
sh£«pe ratio constant of a general person's head. 

35. A person's portrait generation method comprising: 

an image input step of picking up a two-dimen- 
sions: image containing a person's face by us- 
ing an image sensor; 

a head area extracting step of extracting a head 
: . area from Lhe image picked up in the image in- 
put step; 

a feature detection step of detecting position of 
characteristic features of the face by dividing 
the head area thus extracted into face part ar- 
: eas for respective face parts and finding pro- 
jection data of binarized images for the respec- 
tive face parts areas; 

■ a face outline determining step of determining 
a border: between a face outline and a back- 
ground within the head area; and 
:.-an image processing step of creating a per- 
> son's portrait in which the characteristics in the 
' person's face is emphasized based upon the 
- two-dimensional image by using data acquired 
in the head area extraction step, the feature de- 
tection step and the face outline determining 
step. 




36. The person's portrait generation method according 
to claim 35, 

wherein the face parts area is determined by 
relative positional information of the face parts with 
$ respect to the head area preliminarily found. 

37. The person's portrait generation method according 
to claim 35, 

wherein the position of characteristic features 
10 of the face is detected based on a position of a cenl- 

er-of-gravity of the projection data. 

38. The person's portrait generation method according 
to claim 35, 

'5 wherein the position of characteristic features 

of the face is detected by based on a position or a 
maximum value of the projection data. 

39. A person's portrait generation method comprising: 

20 

an image input step of picking up a two-dimen- 
sional image containing a person's face by us- 
ing an image sensor; 

a head area extracting step of extracting a head 
25 area from the image picked up in the image in- 

put step; 

a feature detection step of detecting position ol 
characteristic features within the extracted 
head area; 

30 a face outline determining step of determining 

a border between a face outline and a back- 
ground within the head area by specifying a 
skin color area of the two-dimensional image: 
and 

35 an image processing step of creating a per- 

son's portrait in which the characteristics in the 
person's face is emphasized based upon the 
two-dimensional image by using data acquired 
in the head area extraction step, the feature de- 

40 tection step and the face outline determining 

step. 

40. The person' s portrait generation method according 
to claim 39, 

45 wherein in the face outline determining step, 

the outline of the jaw is determined. 

41 . The person's portrait generation method according 
to claim 39, 

50 wherein in the face outline determining step, 

the skin color area is determined by converting an 
inputted RGB value to an HSV value. 

42. The person* s portrait generation method according 
55 to claim 39, 

wherein the determining method in the face 
outline determining step, with respect to an average 
color in the vicinity of the characteristic features of 
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the face, areas having similar colors are used as 
face area candidates. 

43. A person's portrait generation method comprising: 

an image input step of picking up a two-dimen- 
sional image containing a person's face by us- 
ing an image sensor; 

a head area extracting step of extracting a head 
area frcm the image picked up in the image in- 
put step; 

a feature detection step of detecting position of 
characteristic features of the face for each of 
the face parts within the head area that has 
been extracted; 

a face outline determining step of determining 
a border between a face outline and a back- 
ground Wtthin the head area; and an image 
processing step of creating a person' s portrait 
in which the characteristics in the person's face 
is emphasized by changing the two-dimension- 
al image with respect to each of the face parts. 

44. The person* s portrait generation method according 
to claim 43. 

wherein the image processing step expresses 
a poison' s emotions by changing the shape of a 
paaial Imsge at each of an eye portion, a noise por- 
tion and a mouth portion that are the characteristic 
features of the face. 

45. The person's portrait generation method according 
to claim 43, 

wherein ihe image processing step replaces 
pah ial images at an eye portion, a noise portion and 
a mouth portion that are characteristic features of 
the face. 

46. A communication method using a communication 
terminal thai comprises: 



two-dimensional Image by using data acquired 
in the head area extraction step, the feature de- 
tection step and the face outline determining 
step; and 

5 a step in which, when a communication is re- 

ceived, the person's portrait thus created is dis- 
played so as to inform the user of the receipt ot 
the communication. 

*o 47. A communication method using a communication 
terminal that comprises: 

an image input step of picking up a two-dimen- 
sional Image containing a person's face by us- 
*5 ing an image sensor; 

a head area extracting step of extracting a head 
area from the image picked up in the image in- 
put step; 

a feature detection step of detecting position of 
20 characteristic features of the face by dividing 

the head area thus extracted into face part ar- 
eas for respective face parts and finding pro- 
jection data of binarized images for the respec- 
tive face parts areas; 
25 a face outline determining step of determining 

a border between a face outline and a back- 
ground within the head area; 
an image processing step of creating a per- 
son's portrait in which the characteristics in the 
30 person's face is emphasized based upon the 

two-dimensional image by using data acquired 
in the head area extraction step, the feature de- 
tection step and the face outline determining 
step; and 

35 a step in which, when a communication is re- 

ceived, the person's portrait thus created is dis- 
played so as to inform the user of the receipt of 
the communication. 

40 48. A communication method using a communication 
terminal that comprises: 

an image input step of picking up a two-dimen- 
sional image containing a person's face by us- 
ing an image sensor; 

a head area extracting step of extracting a head 
area from the image picked up in the image in- 
put step; 

a feature detection step of detecting position ol 
characteristic features within the extracted 
head area; 

a face outline determining step of determining 
a border between a face outline and a back- 
ground within the head area by specifying a 
skin color area of the two-dimensional image; 
an image processing step of creating a per- 
son's portrait in which the characteristics in the 
person's face is emphasized based upon the 



an Image input step of picking up a two-dimen- 
sional image containing a person's face by us- 
ing an image sensor; 

a head area extracting step of extracting a head 45 
area from a differential image of a plurality of 
tnetwc-dimensional image picked up in the im- 
age input step; 

a feature detection step of detecting position of 
; characteristic features within the extracted so 
he ad- area; 

a face outline determining step of determining 
a borcijr between a face outline and a back- 

• ground'withih the head area by specifying a 
skin cotor area of the two-dimensional image: 55 
an image' processing step of creating a per- 
son's portrait in which the characteristics In the 

■'■ person's face is emphasized based upon the 
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two-dimensional image by using data acquired 
in the head area extraction step, the feature de- 
tection step and the face outline determining 
step; and 

a step in which, when a communication is re- s 
ceivea\ the person's portrait thus created is dis- 
played so as to inform the user of the receipt of 
the communication. 

49. A communication method using a communication n> 
terminal that comprises: 

an imsge Input step of picking up a two-dimen- 
sional image containing a person's face by us- 
ing an image sensor; 

a head area extracting step of extracting a head 
area fiom the image picked up in the image in- 
put step; 

a feature detection step of detecting position of 
characteristic features of the face for each of 20 
the face parts within the head area that has 
been extracted; 

a face outline determining step of determining 
a border between a face outline and a back- 
ground within the head area; 25 
an image processing step of creating a per- 
son's portrait in which the characteristics in the 
person's face is emphasized by changing the 
two-dimensional image with respect to each of 
the face parts; and 30 
a step in wnich, when a communication is re- 
ceived the person's portrait thus created is dis- 
played so as to inform the user of the receipt of 
the communication. 

35 

50. A r*coraingmedium in which is recorded a person's 
portrait generation program which is executed by a 
computet-/ and comprises: 

e head area extracting program for extracting *o 
a head area of a person from a differential im- 
age of a plurality of two-dimensional images 
picked up by an image input section (1) ; 
a feature detection program for detecting posi- 
tions of characteristic features of the person's 45 
portrait by dividing the head area into face part 
areas' for respective face parts, finding projec- 
tion-data o* hinarized images for the respective 
face parts areas and detecting the position of 
eactrof the face parts based upon a center-of- so 
gravity position of the projection data; 
a face outline determining program for deter- 
mining a border between a face outline and a 
- background within the head area by specifying 
a skin color area of the two-dimensional image; ss 
and 

an image processing program for creating a 
person's portrait in which the characteristics in 



the person's face is emphasized by changing 
the two-dimensional image with respect to each 
of the face parts. 
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